1. Field of the Invention
This invention relates in general to software testing, and in particular to abstracting markup applications in order to expedite and render software testing more flexible and efficient.
2. Description of the Related Art
Software has become an essential part of everyday life worldwide. Many businesses and individuals relay on software to support their daily activities. The quality and reliability of software has never been as important and critical as it is today. Software providers, whether they provide packaged software products or customized solutions sold to businesses, consider the quality of the software they sell a very serious matter. Software defects can severely jeopardize their entire business, especially given the reality that these defects can detrimentally impact customer loyalty and trust.
There are many categories of software applications developed today. One type of application is the markup base application. These applications are defined using markup languages such as, for example, HTML, XHTML, XML, or other similar markup languages. These applications are interpreted using a host that provides the user with elements which the user can use to interact with the application logic. A classic example of these types of applications is a web application. Web applications run in a host like Microsoft's Internet Explorer™ or Mozilla's FireFox™ and interpret an HTML document to present it to the user for interaction.
Many software development companies test their software by compiling a list of test scripts (i.e., test scenarios or test cases) which are run against their software to ensure its quality and to validate its design. A test script must be created for each piece of software to be tested. Some companies hire employees or contractors to run the test manually, while others invest in automating the tests in order to avoid the additional costs associated with the manual labor. There are many advantages to test automation, with a few worth noting such as: 1) automation cuts the costs associated with manual execution, 2) automation cuts down on the execution time, which allows for faster turn around on results, and 3) automation allows for more accurate and consistent results.
With regard to generating automated test scripts for such markup applications, there are increasing challenges and pressures to enhance the efficiency and quality associated with building the test scripts; especially with the increasingly rich content on the web and the wide spread use of dynamic web content, such as ASP.NET or PHP. Some of the key challenges are summarized in the following paragraphs.
First, the ease of building and crafting automated tests which perform the desired tasks is a challenge. Here, in order to execute actions against the software or to verify its content, it would be very beneficial if one could easily and consistently identify elements within the complex markup of the software application under test (i.e., SAUT). For example, some application's output changes dynamically between requests and within the same session. Other applications generate dynamic content with variable element IDs which make it difficult to consistently identify the elements. Moreover, such an environment makes it difficult to automate the test scripts against the application throughout the development cycle.
Second, maintenance and updatability of automated test scripts is a challenge. Depending on how the automated tests are authored, changes to the SAUT can break the functionality of the test scripts because they are unable to adapt to the nature of the SAUT. This, in turn, adds additional costs to repair and update the tests throughout the life cycle of the application and throughout the different future version releases of the application.
There are currently several products on the market that attempt to address the above challenges of automated testing for SAUT, such as, for example QuickTest™ and WinRunner™, which are marketed by Mercury Interaction®. Although these tools have a record and replay functionality to easily craft automated test scripts, they provide little support to help solve the need for flexible identification of elements and loose coupling of test scripts within the SAUT. Thus, each falls short in addressing the maintenance nightmare that comes with such testing solutions. In fact, due to dynamic changes in the software which require re-recording of the scripts from scratch, record and replay testing is one of the most expensive automated testing techniques.
Some other tools like TestComplete™ marketed commercially by AutomatedQA™, attempt to record at the elemental level. This, however, enhances the resilience of the tests, but falls short of addressing issues related to the dynamic nature of certain elements and elements that do not have consistent Properties between users or requests. For example, most markup applications use two very popular methods to identify elements (especially for non-uniform markups like HTML). The first is an element ID or name which defines the element and is globally unique within the SAUT. The second is an element tag name occurrence, which is the occurrence index of a specific tag in the order of the elements appearance within the SAUT.
Information regarding the tag's occurrence or element ID and name is supported by most tools used today. These tools help identify and locate application elements for the automated test scripts. Automated tools like the prior art described above record this information and store it as part of the automated test script to be used later when the script is executed. Although this approach can help in many cases, because its static, it still falls short when the order of elements or the tag's occurrence is dynamic and changes based upon data entered by a user. A good example of this is a table cell in an HTML table. Depending on what data is requested by the user, there might be one cell or multiple cells in a table and, thus, hard coding any specific tag occurrence or element ID is not feasible. In addition to that, dynamic languages like ASP.NET, generate rich content where users in some cases have no control over the attributes of the elements being emitted. For example, ASP.NET has a calendar control which outputs a very complex set of markups that a user cannot control by annotating the elements with IDs or names.
To improve test automation, however, there is a need for adaptable and dependable tools that enable one to more efficiently automate the scenarios that one's software performs in order to generate the automated test scripts. This need has been the key catalyst for innovations in the software development lifecycle and, more specifically, in how software is tested throughout the life cycle. In addition, there is increasing pressure for companies to produce software faster and with lower development and maintenance costs.
Companies have started addressing some of these issues by using abstracted layers that model their specific SAUT built on the top of their automation infrastructure. Automated test scripts go through these abstracted layers to access elements within the application. In the event of a maintenance issue, the use of the abstract model allows the script developer to fix the abstracted layer instead of fixing thousands of automated tests. Although useful, the abstract layers of the conventional systems are hard coded to the specific SAUT. Thus, when any changes are made to the SAUT, the abstract models are rendered useless in most cases. This results in higher development costs due to the constant maintenance required to keep the models in sync with the SAUT. As such, the inventor of the present invention has recognized the need for a new and improved method of automated testing which addresses these shortcomings.